🕷️️ Job Radar • SCRAPING

freelancer.com 2026-04-28 🟠

🔹 Web-scraping solution for competitive e-commerce data
👤 Client: 🇮🇳 palvancha, India Member since 2019-04-29
💰 Price: $23 Average bid
🚩 Problem: Capture detailed product information from competitors' online stores to enhance pricing intelligence and content optimization.
📦 Existing: [URLs of competitor sites]

Specifications:

[Target] - E-commerce product details: price, description, categories, images, variants
[Method] - Python-based web scraping using Scrapy, BeautifulSoup, Selenium or Playwright with anti-bot measures
[UI/UX] - Not applicable
[Stack] - Python (Scrapy, BeautifulSoup, Selenium, Playwright), Requests, Pandas, Git
[Security] - Rotating proxies, polite timing, basic CAPTCHA handling, secure private Git repository
[Format] - CSV or Excel for product data; compressed folder for images

Workflow:

1. Analyze competitor site structure and identify key elements to scrape.
2. Develop a Python script using Scrapy, BeautifulSoup, Selenium, or Playwright with anti-bot measures.
3. Test the scraper on one sample competitor site to ensure full crawl without manual intervention.
4. Export data in CSV/Excel format: product URL, title, description, category, sub-category, price, variant attributes, image links.
5. Download and organize images by SKU into a compressed folder or cloud bucket.
6. Document the scraper with a README explaining rerun instructions, target URL adjustments, and update scheduling.